Anterior Cruciate Ligament Return to Sport after Injury Scale (ACL-RSI) Scores over Time After Anterior Cruciate Ligament Reconstruction: A Systematic Review with Meta-analysis.

BACKGROUND: Psychological readiness is an important consideration for athletes and clinicians when making return to sport decisions following anterior cruciate ligament reconstruction (ACLR). To improve our understanding of the extent of deficits in psychological readiness, a systematic review is necessary. OBJECTIVE: To investigate psychological readiness (measured via the Anterior Cruciate Ligament-Return to Sport after Injury scale (ACL-RSI)) over time after ACL tear and understand if time between injury and surgery, age, and sex are associated with ACL-RSI scores. METHODS: Seven databases were searched from the earliest date available to March 22, 2022. Articles reporting ACL-RSI scores after ACL tear were included. Risk of bias was assessed using the ROBINS-I, RoB-2, and RoBANS tools based on the study design. Evidence certainty was assessed for each analysis. Random-effects meta-analyses pooled ACL-RSI scores, stratified by time post-injury and based on treatment approach (i.e., early ACLR, delayed ACLR, and unclear approach). RESULTS: A total of 83 studies were included in this review (78% high risk of bias). Evidence certainty was 'weak' or 'limited' for all analyses. Overall, ACL-RSI scores were higher at 3 to 6 months post-ACLR (mean = 61.5 [95% confidence interval (CI) 58.6, 64.4], I2 = 94%) compared to pre-ACLR (mean = 44.4 [95% CI 38.2, 50.7], I2 = 98%), remained relatively stable, until they reached the highest point 2 to 5 years after ACLR (mean = 70.7 [95% CI 63.0, 78.5], I2 = 98%). Meta-regression suggests shorter time from injury to surgery, male sex, and older age were associated with higher ACL-RSI scores only 3 to 6 months post-ACLR (heterogeneity explained R2 = 47.6%), and this reduced 1-2 years after ACLR (heterogeneity explained R2 = 27.0%). CONCLUSION: Psychological readiness to return to sport appears to improve early after ACL injury, with little subsequent improvement until ≥ 2-years after ACLR. Longer time from injury to surgery, female sex and older age might be negatively related to ACL-RSI scores 12-24 months after ACLR. Due to the weak evidence quality rating and the considerable importance of psychological readiness for long-term outcomes after ACL injury, there is an urgent need for well-designed studies that maximize internal validity and identify additional prognostic factors for psychological readiness at times critical for return to sport decisions. REGISTRATION: Open Science Framework (OSF), https://osf.io/2tezs/ .

Reverse total shoulder replacement versus anatomical total shoulder replacement for osteoarthritis: population based cohort study using data from the National Joint Registry and Hospital Episode Statistics for England.

OBJECTIVES: To answer a national research priority by comparing the risk-benefit and costs associated with reverse total shoulder replacement (RTSR) and anatomical total shoulder replacement (TSR) in patients having elective primary shoulder replacement for osteoarthritis. DESIGN: Population based cohort study using data from the National Joint Registry and Hospital Episode Statistics for England. SETTING: Public hospitals and publicly funded procedures at private hospitals in England, 2012-20. PARTICIPANTS: Adults aged 60 years or older who underwent RTSR or TSR for osteoarthritis with intact rotator cuff tendons. Patients were identified from the National Joint Registry and linked to NHS Hospital Episode Statistics and civil registration mortality data. Propensity score matching and inverse probability of treatment weighting were used to balance the study groups. MAIN OUTCOME MEASURES: The main outcome measure was revision surgery. Secondary outcome measures included serious adverse events within 90 days, reoperations within 12 months, prolonged hospital stay (more than three nights), change in Oxford Shoulder Score (preoperative to six month postoperative), and lifetime costs to the healthcare service. RESULTS: The propensity score matched population comprised 7124 RTSR or TSR procedures (126 were revised), and the inverse probability of treatment weighted population comprised 12 968 procedures (294 were revised) with a maximum follow-up of 8.75 years. RTSR had a reduced hazard ratio of revision in the first three years (hazard ratio local minimum 0.33, 95% confidence interval 0.18 to 0.59) with no clinically important difference in revision-free restricted mean survival time, and a reduced relative risk of reoperations at 12 months (odds ratio 0.45, 95% confidence interval 0.25 to 0.83) with an absolute risk difference of -0.51% (95% confidence interval -0.89 to -0.13). Serious adverse events and prolonged hospital stay risks, change in Oxford Shoulder Score, and modelled mean lifetime costs were similar. Outcomes remained consistent after weighting. CONCLUSIONS: This study's findings provide reassurance that RTSR is an acceptable alternative to TSR for patients aged 60 years or older with osteoarthritis and intact rotator cuff tendons. Despite a significant difference in the risk profiles of revision surgery over time, no statistically significant and clinically important differences between RTSR and TSR were found in terms of long term revision surgery, serious adverse events, reoperations, prolonged hospital stay, or lifetime healthcare costs.

Open science practices need substantial improvement in prognostic model studies in oncology using machine learning.

OBJECTIVE: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING: We conducted a systematic review, searching the MEDLINE database between December 1, 2022, and December 31, 2022, for studies developing a multivariable prognostic model using machine learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices. RESULTS: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data sharing statements, with 21 (46%) indicating data were available on request to the authors and seven declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their model to be used in practice. The use of reporting guidelines was rare: eight studies (18%) mentioning using a reporting guideline, with 4 (10%) using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis statement, 1 (2%) using Minimum Information About Clinical Artificial Intelligence Modeling and Consolidated Standards Of Reporting Trials-Artificial Intelligence, 1 (2%) using Strengthening The Reporting Of Observational Studies In Epidemiology, 1 (2%) using Standards for Reporting Diagnostic Accuracy Studies, and 1 (2%) using Transparent Reporting of Evaluations with Nonrandomized Designs. CONCLUSION: The adoption of open science principles in oncology studies developing prognostic models using machine learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science are needed for prediction research in oncology.

Directrices para presentación de informes de ensayos clínicos sobre intervenciones con inteligencia artificial: extensión CONSORT-AI / Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension / Diretrizes para relatórios de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão CONSORT-AI

resumen está disponible en el texto completo

ABSTRACT The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

RESUMO A declaração CONSORT 2010 apresenta diretrizes mínimas para relatórios de ensaios clínicos randomizados. Seu uso generalizado tem sido fundamental para garantir a transparência na avaliação de novas intervenções. Recentemente, tem-se reconhecido cada vez mais que intervenções que incluem inteligência artificial (IA) precisam ser submetidas a uma avaliação rigorosa e prospectiva para demonstrar seus impactos sobre os resultados de saúde. A extensão CONSORT-AI (Consolidated Standards of Reporting Trials - Artificial Intelligence) é uma nova diretriz para relatórios de ensaios clínicos que avaliam intervenções com um componente de IA. Ela foi desenvolvida em paralelo à sua declaração complementar para protocolos de ensaios clínicos, a SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials - Artificial Intelligence). Ambas as diretrizes foram desenvolvidas por meio de um processo de consenso em etapas que incluiu revisão da literatura e consultas a especialistas para gerar 29 itens candidatos. Foram feitas consultas sobre esses itens a um grupo internacional composto por 103 interessados diretos, que participaram de uma pesquisa Delphi em duas etapas. Chegou-se a um acordo sobre os itens em uma reunião de consenso que incluiu 31 interessados diretos, e os itens foram refinados por meio de uma lista de verificação piloto que envolveu 34 participantes. A extensão CONSORT-AI inclui 14 itens novos que, devido à sua importância para as intervenções de IA, devem ser informados rotineiramente juntamente com os itens básicos da CONSORT 2010. A CONSORT-AI preconiza que os pesquisadores descrevam claramente a intervenção de IA, incluindo instruções e as habilidades necessárias para seu uso, o contexto no qual a intervenção de IA está inserida, considerações sobre o manuseio dos dados de entrada e saída da intervenção de IA, a interação humano-IA e uma análise dos casos de erro. A CONSORT-AI ajudará a promover a transparência e a integralidade nos relatórios de ensaios clínicos com intervenções que utilizam IA. Seu uso ajudará editores e revisores, bem como leitores em geral, a entender, interpretar e avaliar criticamente a qualidade do desenho do ensaio clínico e o risco de viés nos resultados relatados.

Directrices para los protocolos de ensayos clínicos de intervenciones con inteligencia artificial: la extensión SPIRIT-AI / Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension / Diretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI

resumen está disponible en el texto completo

ABSTRACT The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

RESUMO A declaração SPIRIT 2013 tem como objetivo melhorar a integralidade dos relatórios dos protocolos de ensaios clínicos, fornecendo recomendações baseadas em evidências para o conjunto mínimo de itens que devem ser abordados. Essas orientações têm sido fundamentais para promover uma avaliação transparente de novas intervenções. Recentemente, tem-se reconhecido cada vez mais que intervenções que incluem inteligência artificial (IA) precisam ser submetidas a uma avaliação rigorosa e prospectiva para demonstrar seus impactos sobre os resultados de saúde. A extensão SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials - Artificial Intelligence) é uma nova diretriz de relatório para protocolos de ensaios clínicos que avaliam intervenções com um componente de IA. Essa diretriz foi desenvolvida em paralelo à sua declaração complementar para relatórios de ensaios clínicos, CONSORT-AI (Consolidated Standards of Reporting Trials - Artificial Intelligence). Ambas as diretrizes foram desenvolvidas por meio de um processo de consenso em etapas que incluiu revisão da literatura e consultas a especialistas para gerar 26 itens candidatos. Foram feitas consultas sobre esses itens a um grupo internacional composto por 103 interessados diretos, que participaram de uma pesquisa Delphi em duas etapas. Chegou-se a um acordo sobre os itens em uma reunião de consenso que incluiu 31 interessados diretos, e os itens foram refinados por meio de uma lista de verificação piloto que envolveu 34 participantes. A extensão SPIRIT-AI inclui 15 itens novos que foram considerados suficientemente importantes para os protocolos de ensaios clínicos com intervenções que utilizam IA. Esses itens novos devem constar dos relatórios de rotina, juntamente com os itens básicos da SPIRIT 2013. A SPIRIT-AI preconiza que os pesquisadores descrevam claramente a intervenção de IA, incluindo instruções e as habilidades necessárias para seu uso, o contexto no qual a intervenção de IA será integrada, considerações sobre o manuseio dos dados de entrada e saída, a interação humano-IA e a análise de casos de erro. A SPIRIT-AI ajudará a promover a transparência e a integralidade nos protocolos de ensaios clínicos com intervenções que utilizam IA. Seu uso ajudará editores e revisores, bem como leitores em geral, a entender, interpretar e avaliar criticamente o delineamento e o risco de viés de um futuro estudo clínico.

Shoulder replacement surgery's rising demand, inequality of provision, and variation in outcomes: cohort study using Hospital Episode Statistics for England.

BACKGROUND: The aim of this study was to forecast future patient demand for shoulder replacement surgery in England and investigate any geographic and socioeconomic inequalities in service provision and patient outcomes. METHODS: For this cohort study, all elective shoulder replacements carried out by NHS hospitals and NHS-funded care in England from 1999 to 2020 were identified using Hospital Episode Statistics data. Eligible patients were aged 18 years and older. Shoulder replacements for malignancy or acute trauma were excluded. Population estimates and projections were obtained from the Office for National Statistics. Standardised incidence rates and the risks of serious adverse events (SAEs) and revision surgery were calculated and stratified by geographical region, socioeconomic deprivation, sex, and age band. Hospital costs for each admission were calculated using Healthcare Resource Group codes and NHS Reference Costs based on the National Reimbursement System. Projected rates and hospital costs were predicted until the year 2050 for two scenarios of future growth. RESULTS: A total of 77,613 elective primary and 5847 revision shoulder replacements were available for analysis. Between 1999 and 2020, the standardised incidence of primary shoulder replacements in England quadrupled from 2.6 to 10.4 per 100,000 population, increasing predominantly in patients aged over 65 years. As many as 1 in 6 patients needed to travel to a different region for their surgery indicating inequality of service provision. A temporal increase in SAEs was observed: the 30-day risk increased from 1.3 to 4.8% and the 90-day risk increased from 2.4 to 6.0%. Patients from the more deprived socioeconomic groups appeared to have a higher risk of SAEs and revision surgery. Shoulder replacements are forecast to increase by up to 234% by 2050 in England, reaching 20,912 procedures per year with an associated annual cost to hospitals of £235 million. CONCLUSIONS: This study reports a rising incidence of shoulder replacements, regional disparities in service provision, and an overall increasing risk of SAEs, especially in more deprived socioeconomic groups. These findings highlight the need for better healthcare planning to match local population demand, while more research is needed to understand and prevent the increase observed in SAEs.

A framework for the definition and interpretation of the use of surrogate endpoints in interventional trials.

Background: Interventional trials that evaluate treatment effects using surrogate endpoints have become increasingly common. This paper describes four linked empirical studies and the development of a framework for defining, interpreting and reporting surrogate endpoints in trials. Methods: As part of developing the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) extensions for randomised trials reporting surrogate endpoints, we undertook a scoping review, e-Delphi study, consensus meeting, and a web survey to examine current definitions and stakeholder (including clinicians, trial investigators, patients and public partners, journal editors, and health technology experts) interpretations of surrogate endpoints as primary outcome measures in trials. Findings: Current surrogate endpoint definitional frameworks are inconsistent and unclear. Surrogate endpoints are used in trials as a substitute of the treatment effects of an intervention on the target outcome(s) of ultimate interest, events measuring how patients feel, function, or survive. Traditionally the consideration of surrogate endpoints in trials has focused on biomarkers (e.g., HDL cholesterol, blood pressure, tumour response), especially in the medical product regulatory setting. Nevertheless, the concept of surrogacy in trials is potentially broader. Intermediate outcomes that include a measure of function or symptoms (e.g., angina frequency, exercise tolerance) can also be used as substitute for target outcomes (e.g., all-cause mortality)-thereby acting as surrogate endpoints. However, we found a lack of consensus among stakeholders on accepting and interpreting intermediate outcomes in trials as surrogate endpoints or target outcomes. In our assessment, patients and health technology assessment experts appeared more likely to consider intermediate outcomes to be surrogate endpoints than clinicians and regulators. Interpretation: There is an urgent need for better understanding and reporting on the use of surrogate endpoints, especially in the setting of interventional trials. We provide a framework for the definition of surrogate endpoints (biomarkers and intermediate outcomes) and target outcomes in trials to improve future reporting and aid stakeholders' interpretation and use of trial surrogate endpoint evidence. Funding: SPIRIT-SURROGATE/CONSORT-SURROGATE project is Medical Research Council Better Research Better Health (MR/V038400/1) funded.

Predicting 10-year breast cancer mortality risk in the general female population in England: a model development and validation study.

BACKGROUND: Identifying female individuals at highest risk of developing life-threatening breast cancers could inform novel stratified early detection and prevention strategies to reduce breast cancer mortality, rather than only considering cancer incidence. We aimed to develop a prognostic model that accurately predicts the 10-year risk of breast cancer mortality in female individuals without breast cancer at baseline. METHODS: In this model development and validation study, we used an open cohort study from the QResearch primary care database, which was linked to secondary care and national cancer and mortality registers in England, UK. The data extracted were from female individuals aged 20-90 years without previous breast cancer or ductal carcinoma in situ who entered the cohort between Jan 1, 2000, and Dec 31, 2020. The primary outcome was breast cancer-related death, which was assessed in the full dataset. Cox proportional hazards, competing risks regression, XGBoost, and neural network modelling approaches were used to predict the risk of breast cancer death within 10 years using routinely collected health-care data. Death due to causes other than breast cancer was the competing risk. Internal-external validation was used to evaluate prognostic model performance (using Harrell's C, calibration slope, and calibration in the large), performance heterogeneity, and transportability. Internal-external validation involved dataset partitioning by time period and geographical region. Decision curve analysis was used to assess clinical utility. FINDINGS: We identified data for 11 626 969 female individuals, with 70 095 574 person-years of follow-up. There were 142 712 (1·2%) diagnoses of breast cancer, 24 043 (0·2%) breast cancer-related deaths, and 696 106 (6·0%) deaths from other causes. Meta-analysis pooled estimates of Harrell's C were highest for the competing risks model (0·932, 95% CI 0·917-0·946). The competing risks model was well calibrated overall (slope 1·011, 95% CI 0·978-1·044), and across different ethnic groups. Decision curve analysis suggested favourable clinical utility across all age groups. The XGBoost and neural network models had variable performance across age and ethnic groups. INTERPRETATION: A model that predicts the combined risk of developing and then dying from breast cancer at the population level could inform stratified screening or chemoprevention strategies. Further evaluation of the competing risks model should comprise effect and health economic assessment of model-informed strategies. FUNDING: Cancer Research UK.

The estimand framework had implications in time to patient-reported outcomes deterioration analyses in cancer clinical trials.

OBJECTIVES: To apply the estimand framework in time to deterioration (TTD) analysis of patient-reported outcomes (PROs), and identify the appropriate statistical methods to deal with intercurrent event (IEs) such as death. STUDY DESIGN AND SETTING: Data from phase II randomized trial were used. We estimated TTD using European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-C30 questionnaire with death as the IE, by applying Kaplan-Meier (K.M.) estimator and Cox proportional hazards (PH) model. The Fine-Gray approach was explored, accounting for death as a competing risk. The estimands targeted by the aforementioned methods were defined. RESULTS: We analyzed the data of 64 patients with available questionnaires at baseline. The most notable differences in TTD estimates were observed for deterioration in physical functioning: the hazard ratios were 0.44 [95% CI 0.22-0.90] and 0.62 [95% CI 0.36-1.07] by either ignoring death (31 events) or considering it as deterioration (58 events), respectively (Cox-PH model). When considering death as a competing event (Fine-Gray model), the sub-HRs was 0.51 [95% CI 0.26-1.01]. CONCLUSION: Depending on the proportion and distribution of deaths occurring before deterioration between arms, the Fine-Gray competing risks model should be considered rather than KM estimator and Cox PH model to reflect the patient's experience of the disease and treatment burden.

Artificial intelligence in lung cancer diagnostic imaging: a review of the reporting and conduct of research published 2018-2019.

Objective: This study aimed to describe the methodologies used to develop and evaluate models that use artificial intelligence (AI) to analyse lung images in order to detect, segment (outline borders of), or classify pulmonary nodules as benign or malignant. Methods: In October 2019, we systematically searched the literature for original studies published between 2018 and 2019 that described prediction models using AI to evaluate human pulmonary nodules on diagnostic chest images. Two evaluators independently extracted information from studies, such as study aims, sample size, AI type, patient characteristics, and performance. We summarised data descriptively. Results: The review included 153 studies: 136 (89%) development-only studies, 12 (8%) development and validation, and 5 (3%) validation-only. CT scans were the most common type of image type used (83%), often acquired from public databases (58%). Eight studies (5%) compared model outputs with biopsy results. 41 studies (26.8%) reported patient characteristics. The models were based on different units of analysis, such as patients, images, nodules, or image slices or patches. Conclusion: The methods used to develop and evaluate prediction models using AI to detect, segment, or classify pulmonary nodules in medical imaging vary, are poorly reported, and therefore difficult to evaluate. Transparent and complete reporting of methods, results and code would fill the gaps in information we observed in the study publications. Advances in knowledge: We reviewed the methodology of AI models detecting nodules on lung images and found that the models were poorly reported and had no description of patient characteristics, with just a few comparing models' outputs with biopsies results. When lung biopsy is not available, lung-RADS could help standardise the comparisons between the human radiologist and the machine. The field of radiology should not give up principles from the diagnostic accuracy studies, such as the choice for the correct ground truth, just because AI is used. Clear and complete reporting of the reference standard used would help radiologists trust in the performance that AI models claim to have. This review presents clear recommendations about the essential methodological aspects of diagnostic models that should be incorporated in studies using AI to help detect or segmentate lung nodules. The manuscript also reinforces the need for more complete and transparent reporting, which can be helped using the recommended reporting guidelines.

Association between surgeon volume and patient outcomes after elective shoulder replacement surgery using data from the National Joint Registry and Hospital Episode Statistics for England: population based cohort study.

OBJECTIVE: To investigate the association between surgeon volume and patient outcomes after elective shoulder replacement surgery to improve patient outcomes and inform future resource planning for joint replacement surgery. DESIGN: Prospective cohort study. SETTING: Public and private hospitals in the United Kingdom, 2012-20. PARTICIPANTS: Adults aged 18 years or older who had shoulder replacement surgery, identified in the National Joint Registry, with linkage of participants in England to Hospital Episode Statistics data. MAIN OUTCOME MEASURES: The main outcome measure was revision surgery. Secondary outcome measures were reoperation within 12 months, serious adverse events, and prolonged hospital stay (>3 nights) after shoulder replacement surgery. RESULTS: 39 281 shoulder replacement procedures undertaken by 638 consultant surgeons at 416 surgical units met the inclusion criteria and were available for analysis. Multilevel mixed effects models and restricted cubic splines were fit to examine the association between a surgeon's mean annual volume and risk of adverse patient outcomes, with a minimum volume threshold of 10.4 procedures yearly identified. Below this threshold the risk of revision surgery was significantly increased, as much as twice that of surgeons with the lowest risk (hazard ratio 1.94, 95% confidence interval 1.27 to 2.97). A greater mean annual surgical volume was also associated with a significantly lower risk of reoperations, fewer serious adverse events, and shorter hospital stay, with no thresholds identified. Annual variation in surgeon volume was not associated with any of the outcomes assessed. CONCLUSIONS: In the healthcare system represented by these registry data, an association was found between surgeons who averaged more than 10.4 shoulder replacements yearly and lower rates of revision surgery and reoperation, lower risk of serious adverse events, and shorter hospital stays. These findings should inform resource planning for surgical services and joint replacement surgery waiting lists and improve patient outcomes after shoulder replacement surgery.

Systematic review highlights high risk of bias of clinical prediction models for blood transfusion in patients undergoing elective surgery.

BACKGROUND: Blood transfusion can be a lifesaving intervention after perioperative blood loss. Many prediction models have been developed to identify patients most likely to require blood transfusion during elective surgery, but it is unclear whether any are suitable for clinical practice. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE, Embase, PubMed, The Cochrane Library, Transfusion Evidence Library, Scopus, and Web of Science databases for studies reporting the development or validation of a blood transfusion prediction model in elective surgery patients between January 1, 2000 and June 30, 2021. We extracted study characteristics, discrimination performance (c-statistics) of final models, and data, which we used to perform risk of bias assessment using the Prediction model risk of bias assessment tool (PROBAST). RESULTS: We reviewed 66 studies (72 developed and 48 externally validated models). Pooled c-statistics of externally validated models ranged from 0.67 to 0.78. Most developed and validated models were at high risk of bias due to handling of predictors, validation methods, and too small sample sizes. CONCLUSION: Most blood transfusion prediction models are at high risk of bias and suffer from poor reporting and methodological quality, which must be addressed before they can be safely used in clinical practice.

Development and internal-external validation of statistical and machine learning models for breast cancer prognostication: cohort study.

OBJECTIVE: To develop a clinically useful model that estimates the 10 year risk of breast cancer related mortality in women (self-reported female sex) with breast cancer of any stage, comparing results from regression and machine learning approaches. DESIGN: Population based cohort study. SETTING: QResearch primary care database in England, with individual level linkage to the national cancer registry, Hospital Episodes Statistics, and national mortality registers. PARTICIPANTS: 141 765 women aged 20 years and older with a diagnosis of invasive breast cancer between 1 January 2000 and 31 December 2020. MAIN OUTCOME MEASURES: Four model building strategies comprising two regression (Cox proportional hazards and competing risks regression) and two machine learning (XGBoost and an artificial neural network) approaches. Internal-external cross validation was used for model evaluation. Random effects meta-analysis that pooled estimates of discrimination and calibration metrics, calibration plots, and decision curve analysis were used to assess model performance, transportability, and clinical utility. RESULTS: During a median 4.16 years (interquartile range 1.76-8.26) of follow-up, 21 688 breast cancer related deaths and 11 454 deaths from other causes occurred. Restricting to 10 years maximum follow-up from breast cancer diagnosis, 20 367 breast cancer related deaths occurred during a total of 688 564.81 person years. The crude breast cancer mortality rate was 295.79 per 10 000 person years (95% confidence interval 291.75 to 299.88). Predictors varied for each regression model, but both Cox and competing risks models included age at diagnosis, body mass index, smoking status, route to diagnosis, hormone receptor status, cancer stage, and grade of breast cancer. The Cox model's random effects meta-analysis pooled estimate for Harrell's C index was the highest of any model at 0.858 (95% confidence interval 0.853 to 0.864, and 95% prediction interval 0.843 to 0.873). It appeared acceptably calibrated on calibration plots. The competing risks regression model had good discrimination: pooled Harrell's C index 0.849 (0.839 to 0.859, and 0.821 to 0.876, and evidence of systematic miscalibration on summary metrics was lacking. The machine learning models had acceptable discrimination overall (Harrell's C index: XGBoost 0.821 (0.813 to 0.828, and 0.805 to 0.837); neural network 0.847 (0.835 to 0.858, and 0.816 to 0.878)), but had more complex patterns of miscalibration and more variable regional and stage specific performance. Decision curve analysis suggested that the Cox and competing risks regression models tested may have higher clinical utility than the two machine learning approaches. CONCLUSION: In women with breast cancer of any stage, using the predictors available in this dataset, regression based methods had better and more consistent performance compared with machine learning approaches and may be worthy of further evaluation for potential clinical use, such as for stratified follow-up.

Bibliometric analysis of academic journal recommendations and requirements for surgical and anesthesiologic adverse events reporting.

BACKGROUND: Standards for reporting surgical adverse events (AEs) vary widely within the scientific literature. Failure to adequately capture AEs hinders efforts to measure the safety of healthcare delivery and improve the quality of care. The aim of the present study is to assess the prevalence and typology of perioperative AE reporting guidelines among surgery and anesthesiology journals. MATERIALS AND METHODS: In November 2021, three independent reviewers queried journal lists from the SCImago Journal & Country Rank (SJR) portal (www.scimagojr.com), a bibliometric indicator database for surgery and anesthesiology academic journals. Journal characteristics were summarized using SCImago, a bibliometric indicator database extracted from Scopus journal data. Quartile 1 (Q1) was considered the top quartile and Q4 bottom quartile based on the journal impact factor. Journal author guidelines were collected to determine whether AE reporting recommendations were included and, if so, the preferred reporting procedures. RESULTS: Of 1409 journals queried, 655 (46.5%) recommended surgical AE reporting. Journals most likely to recommend AE reporting were: by category surgery (59.1%), urology (53.3%), and anesthesia (52.3%); in top SJR quartiles (i.e. more influential); by region, based in Western Europe (49.8%), North America (49.3%), and the Middle East (48.3%). CONCLUSIONS: Surgery and anesthesiology journals do not consistently require or provide recommendations on perioperative AE reporting. Journal guidelines regarding AE reporting should be standardized and are needed to improve the quality of surgical AE reporting with the ultimate goal of improving patient morbidity and mortality.

Prediction Models for Bronchopulmonary Dysplasia in Preterm Infants: A Systematic Review and Meta-Analysis.

OBJECTIVE: To review systematically and assess the accuracy of prediction models for bronchopulmonary dysplasia (BPD) at 36 weeks of postmenstrual age. STUDY DESIGN: Searches were conducted in MEDLINE and EMBASE. Studies published between 1990 and 2022 were included if they developed or validated a prediction model for BPD or the combined outcome death/BPD at 36 weeks in the first 14 days of life in infants born preterm. Data were extracted independently by 2 authors following the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (ie, CHARMS) and PRISMA guidelines. Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (ie, PROBAST). RESULTS: Sixty-five studies were reviewed, including 158 development and 108 externally validated models. Median c-statistic of 0.84 (range 0.43-1.00) was reported at model development, and 0.77 (range 0.41-0.97) at external validation. All models were rated at high risk of bias, due to limitations in the analysis part. Meta-analysis of the validated models revealed increased c-statistics after the first week of life for both the BPD and death/BPD outcome. CONCLUSIONS: Although BPD prediction models perform satisfactorily, they were all at high risk of bias. Methodologic improvement and complete reporting are needed before they can be considered for use in clinical practice. Future research should aim to validate and update existing models.

Multivariable prediction models for atrial fibrillation after cardiac surgery: a systematic review protocol.

INTRODUCTION: Dozens of multivariable prediction models for atrial fibrillation after cardiac surgery (AFACS) have been published, but none have been incorporated into regular clinical practice. One of the reasons for this lack of adoption is poor model performance due to methodological weaknesses in model development. In addition, there has been little external validation of these existing models to evaluate their reproducibility and transportability. The aim of this systematic review is to critically appraise the methodology and risk of bias of papers presenting the development and/or validation of models for AFACS. METHODS: We will identify studies that present the development and/or validation of a multivariable prediction model for AFACS through searches of PubMed, Embase and Web of Science from inception to 31 December 2021. Pairs of reviewers will independently extract model performance measures, assess methodological quality and assess risk of bias of included studies using extraction forms adapted from a combination of the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist and the Prediction Model Risk of Bias Assessment Tool. Extracted information will be reported by narrative synthesis and descriptive statistics. ETHICS AND DISSEMINATION: This systemic review will only include published aggregate data, so no protected health information will be used. Study findings will be disseminated through peer-reviewed publications and scientific conference presentations. Further, this review will identify weaknesses in past AFACS prediction model development and validation methodology so that subsequent studies can improve upon prior practices and produce a clinically useful risk estimation tool. PROSPERO REGISTRATION NUMBER: CRD42019127329.

Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review.

OBJECTIVES: In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS: We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION: The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.

Minimum sample size for developing a multivariable prediction model using multinomial logistic regression.

AIMS: Multinomial logistic regression models allow one to predict the risk of a categorical outcome with > 2 categories. When developing such a model, researchers should ensure the number of participants (n) is appropriate relative to the number of events (Ek) and the number of predictor parameters (pk) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes. PROPOSED CRITERIA: The first criterion aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R2 Nagelkerke. The third criterion aims to ensure the overall risk is estimated precisely. For criterion (i), we show the sample size must be based on the anticipated Cox-snell R2 of distinct 'one-to-one' logistic regression models corresponding to the sub-models of the multinomial logistic regression, rather than on the overall Cox-snell R2 of the multinomial logistic regression. EVALUATION OF CRITERIA: We tested the performance of the proposed criteria (i) through a simulation study and found that it resulted in the desired level of overfitting. Criterion (ii) and (iii) were natural extensions from previously proposed criteria for binary outcomes and did not require evaluation through simulation. SUMMARY: We illustrated how to implement the sample size criteria through a worked example considering the development of a multinomial risk prediction model for tumour type when presented with an ovarian mass. Code is provided for the simulation and worked example. We will embed our proposed criteria within the pmsampsize R library and Stata modules.

Prognostic model for atrial fibrillation after cardiac surgery: a UK cohort study.

OBJECTIVE: To develop a validated clinical prognostic model to determine the risk of atrial fibrillation after cardiac surgery as part of the PARADISE project (NIHR131227). METHODS: Prospective cohort study with linked electronic health records from a cohort of 5.6 million people in the United Kingdom Clinical Practice Research Datalink from 1998 to 2016. For model development, we considered a priori candidate predictors including demographics, medical history, medications, and clinical biomarkers. We evaluated associations between covariates and the AF incidence at the end of follow-up using logistic regression with the least absolute shrinkage and selection operator. The model was validated internally with the bootstrap method; subsequent performance was examined by discrimination quantified with the c-statistic and calibration assessed by calibration plots. The study follows TRIPOD guidelines. RESULTS: Between 1998 and 2016, 33,464 patients received cardiac surgery among the 5,601,803 eligible individuals. The final model included 13-predictors at baseline: age, year of index surgery, elevated CHA2DS2-VASc score, congestive heart failure, hypertension, acute coronary syndromes, mitral valve disease, ventricular tachycardia, valve surgery, receiving two combined procedures (e.g., valve replacement + coronary artery bypass grafting), or three combined procedures in the index procedure, statin use, and ethnicity other than white or black (statins and ethnicity were protective). This model had an optimism-corrected C-statistic of 0.68 both for the derivation and validation cohort. Calibration was good. CONCLUSIONS: We developed a model to identify a group of individuals at high risk of AF and adverse outcomes who could benefit from long-term arrhythmia monitoring, risk factor management, rhythm control and/or thromboprophylaxis.